Method and apparatus for detecting fraud attempts in reverse vending machines

ABSTRACT

A reverse vending machine, including: a chamber adapted to receive an object returned to the reverse vending machine; a plurality of cameras arranged around the perimeter of the chamber for viewing said object; a transparent or translucent plate arranged such that the cameras in use view the object obliquely through the transparent or translucent plate; and means adapted to couple light into the plate such that the light undergoes total internal reflection in the plate. Also, a method of detecting dirt in a reverse vending machine.

FIELD OF USE

The present invention relates to reverse vending machines, and inparticular to fraud detection in reverse vending machines.

BACKGROUND

Systems for recycling of returnable containers, for example beveragecontainers, have been in place for many years as a means to preventlittering and conserve resources. Such systems were originally based onmanual handling of returned containers by vendors, but the process hasbeen made more efficient by the introduction of reverse vending machineswhich were able to accept empty containers, verify the authenticity ofthe container, and issue a receipt that can be exchanged for cash orused as payment in the store where the reverse vending machine islocated.

Over the years reverse vending machines have been made more efficientand sophisticated. They are now typically able to handle a wide range ofcontainers made from various materials, most often glass, PET(polyethylene terephthalate), steel and aluminum. A reverse vendingmachine is typically able to receive the containers, validate them basedon shape and other physical characteristics as well as bar codes andother markings, and sort them based on material or type. Some machinesare able to store reusable containers while containers that are onlyrecycled for their material are crushed and stored separately. A reversevending machine should be able to reject non-returnable containers,detect and handle fraud attempts and assign the proper deposit returnvalue to a wide range of containers. At the same time a machine must bereliable and regular maintenance should not be work intensive or requirefrequent replacement of parts.

Consequently, there is a need for constant improvement of reversevending machines in order to meet these challenges as well as newchallenges resulting from for example introduction of new types ofreturnable containers, and more sophisticated fraud attempts.

SUMMARY

The invention relates to methods and apparatuses for detecting fraudattempts in reverse vending machines. The invention uses featuredetection in images and optionally data from other sensors to detectinconsistencies relative to the expected behavior and appearance of areturned item inside a reverse vending machine. In particular theinvention applies two different feature extraction algorithms to imagedata and sensor data in order to detect inconsistent positions ofobjects inside the reverse vending machine. In the various embodimentsof the invention the two different feature extraction algorithms ormethods may extract features that are representative of or associatedwith the same object two different objects. The feature extractionalgorithms may also be applied to data obtained from the same sensor orfrom different sensors and at the same point in time or at differentpoints in time.

Accordingly, in a first aspect of the invention a method is provided. Inorder to perform the method, a reverse vending machine may comprise atleast one image sensor and at least one data processing unit. The atleast one image sensor may be used to obtain a set of one or more imagesof a returned item inserted into the reverse vending machine. At leastone data processing unit may then apply a first feature extractionalgorithm to extract a first feature from at least a first subset of theset of one or more images and a second feature extraction algorithm toextract a second feature from at least a second subset of the set of oneor more images. Alternatively the second feature extraction algorithmcan be applied to data received from an additional sensor. Using thedata processing unit, it may then be determined whether a position or amotion of said first feature is inconsistent with a position or motionof said second feature.

In embodiments using additional sensors that are not image sensors, thesecond feature extraction algorithm may be applied to data received froman ultrasonic sensing device included in the reverse vending machine.Another alternative is to use a laser ranging device included in thereverse vending machine.

In some embodiments of the invention, the set of one or more imagesincludes at least two images obtained at least two different points intime, for example in order to detect inconsistent motion over time.

In one embodiment, the first feature extraction algorithm is used toextract a feature from an image obtained at a first point in time, thefeature being indicative of a relative position of an object at saidfirst point in time, and the second feature extraction algorithm is usedto extract a feature from an image obtained at a second point in time,the feature being indicative of a relative position of said object atsaid second point in time. The determining may then include estimating avelocity of the object based on the relative positions and the timeinterval between the first and the second point in time, and comparingthe estimated velocity with an expected velocity. It should be notedthat according to this embodiment, while the feature extractionalgorithms and the extracted features are not the same, they are assumedto be associated with the same object inside the reverse vendingmachine.

The expected velocity in this embodiment may be any velocity which ispositive relative to the direction a returnable item normally travelsthrough the reverse vending machine, and the determining may indicateinconsistent motion if the estimated velocity is a velocity which isnegative relative to the direction a returnable item normally travelsthrough the reverse vending machine.

Alternatively, the expected velocity may be based on the velocity atwhich a conveyor arranged to transport returned items through thereverse vending machine is operating, and the determining may indicateinconsistent motion if the estimated velocity is substantially differentfrom the expected velocity.

According to yet another alternative, the expected velocity is based ona determined estimated velocity resulting from the application of thefirst feature extraction, the second feature extraction and thedetermining on a previous set of images, and the method may then furthercomprise using the result of the determining to update the expectedvelocity prior to the application of the first feature extraction, thesecond feature extraction and the determining on a subsequent set ofimages.

According to one embodiment of the invention, the first featureextraction algorithm includes applying edge detection along two axesand, based on the edge detection, identifying a region of interest insaid first image. The second feature extraction algorithm may theninclude searching for a region in the second image which is highlycorrelated with said region of interest in the first image. Theinvention does not require that the second extraction algorithm isapplied to and image that has been obtained after the first image, andinstead the second image may have been obtained at a previous point intime.

According to an alternative embodiment, the first feature extractionalgorithm is used to extract a first feature indicative of a relativeposition of a first object associated with the first feature from atleast two images obtained at least two different points in time. Thesecond feature extraction algorithms is then used to extract a secondfeature indicative of a relative position of a second object associatedwith the second feature from at least two images obtained at least twodifferent points in time. The determining may then include estimatingrelative velocities for the first and the second object based on therelative positions and time intervals between the different points intime, and indicating an inconsistent motion of the first object relativeto the second object if there is a substantial difference between saidfirst and said second estimated velocity.

Such a determination may for example indicate that an object that shouldbe attached to the returnable item is instead moving independently ofthe returnable item inside the reverse vending machine. For example, thefirst feature extraction algorithm can be applied to images obtainedfrom a first image sensor which is part of a barcode reader and thesecond feature extraction algorithm can be applied to images obtainedfrom a second image sensor which is part of a shape detection camera.The first feature will then be an image feature which is representativeof a barcode and the second feature is representative of a shape of areturned item.

In yet another embodiment, the first feature extraction algorithmincludes detecting the presence and position of a thin object in atleast one image from the first subset of images and the second featureextraction algorithm includes detecting a position of a returned itemfrom at least one image from the second subset of images or from othersensor data. The determining may then include comparing the position ofthe detected thin object relative to the position of the returned item.

According to this embodiment, detecting the presence and position of athin object may include using a foreground-background segmentationmethod to generate a processed image, classifying each pixel in theprocessed image as either an object pixel or a background pixel, andclassifying each object pixel as either “normal” or “thin”. Theforeground-background segmentation method may for example includedividing a current image by a reference image.

According to another embodiment, the first feature extraction algorithmincludes detecting the presence and position of at least onecharacteristic feature of the shape of a returned item from at least twoimages in the first set of images, the at least two images beingobtained at different points in time. The second feature extractionalgorithm includes detecting a position of a returned item from at leasttwo images from the second subset of images or from other sensor data.The determining may then indicate an inconsistency if the characteristicfeature has a relative position with respect to the overall shape of thereturned item which differs substantially in the at least two imagesfrom the first set of images.

Detecting the presence and position of at least one characteristicfeature of the shape of a returned item may include using aforeground-background segmentation method to generate a processed image,classifying each pixel in said processed image as either an object pixelor a background pixel, and detecting a feature as a peak in the 1stderivative of a curve tracing the edge between object pixels andbackground pixels. This type of feature detection may, for example,detect fingers or other objects holding or being attached to thereturned item, and then starting to move independently from the returnedobject.

In some embodiments of the invention, the first subset of images areobtained using an image sensor which is configured to view a returnedobject silhouetted against a background including a retroreflector, andwhere a light source and a beam splitter are configured such that thelight source and the image sensor are optically co-located as seen fromthe retroreflector.

According to another aspect of the invention, a reverse vending machineis provided. The reverse vending machine may include an opening throughwhich returnable items can be inserted into the interior of the machine,at least one image sensor configured to obtain images of at least asection of the interior, and at least one data processing unit.Furthermore, the reverse vending machine may comprise at least one dataprocessing unit configured to receive a set of one or more images of areturned item inserted into the reverse vending machine from the atleast one camera, at least one feature extraction unit configured toapply a first feature extraction algorithm to extract a first featurefrom at least a first subset of the set of one or more images and applya second feature extraction algorithm to extract a second feature fromat least a second subset of said set of one or more images.Alternatively, the second feature extraction algorithm may be applied todata received by the at least one data processing unit from anadditional sensor. At least one data processing unit (i.e. the same dataprocessing unit or a different data processing unit, or several dataprocessing units operating in parallel) may be configured to determinewhether a position or a motion of the first feature is inconsistent witha position or a motion of the second feature.

The reverse vending machine may further comprise an ultrasonic sensorconfigured to obtain sensor data from inside the interior and providedata to the at least one data processing unit. The feature extractionunit may then be configured to apply the second feature extractionalgorithm to data received from the ultrasonic sensor. The reversevending machine may also be provided with a laser ranging device insteadof or in addition to an ultrasonic sensor. Either of these types ofsensors may be used to estimate the position or motion of a returneditem inside the reverse vending machine for comparison with the positionor motion of some other object that may or may not be part of thereturned item.

In order to determine or estimate motion (or changes in position overtime), at least one data processing unit may be configured to controlthe at least one camera to obtain images at at least two differentpoints in time.

According to one embodiment, at least one feature extraction unit isconfigured to apply the first feature extraction algorithm to an imageobtained at a first point in time and to apply the second featureextraction algorithm to an image obtained at a second point in time. Atleast one data processing unit may then be configured to estimate avelocity of an object present in said interior based on the relativepositions of the first and the second features, and to compare theestimated velocity with an expected velocity.

According to another embodiment, the at least one feature extractionunit is configured to apply said first feature extraction algorithm andsaid second feature extraction algorithm to images obtained at least twodifferent points in time. At least one said processor may then beconfigured to estimate relative velocities for a first and a secondobject associated with said first and said second feature, respectively,and to indicate an inconsistent motion of the first object relative tothe second object if there is a substantial difference between the firstand the second estimated velocity.

In some embodiments of the invention, the reverse vending machineincludes at least one image sensor with a relatively narrow field ofview in order to capture images of barcodes attached to returned itemsat least partly present in said interior, and at least one second imagesensor with a relatively wide field of view in order to capture imagesof the shape of returned items at least partly present in said interior.The returned item does, of course, not have to be fully present in theinterior. As long as at least a part of the item can be viewed by atleast one camera it does not necessarily matter if the item is entirelypresent, only partly present, moving into or moving out of the interior.Instead, this may be left as a design choice depending on the needs in aparticular embodiment.

In some embodiments of the invention the reverse vending may furthercomprise a retroreflector, a beam splitter and a light source. At leastone image sensor may then be configured to view a returned objectsilhouetted against the retroreflector, and the light source and thebeam splitter may be configured such that the light source and saidimage sensor are optically co-located as seen from the retroreflector.

Consistent with the principles of the invention, the various componentsof the reverse vending machine may be implemented in different ways. Forexample, the feature extraction unit may be a set of computer programinstructions stored in a storage device and executable by the at leastone data processing unit to perform the various feature extractionoperations. Alternatively, the feature extraction unit may beimplemented as a Field Programmable Gate Array (FPGA) or as anApplication Specific Integrated Circuit (ASIC). The reverse vendingmachine may, of course, include several feature extraction units.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows a reverse vending machine consistent with the principles ofthe invention;

FIG. 2 shows a cross sectional view of the interior of a reverse vendingmachine;

FIG. 3 is a flowchart illustrating the various steps of a methodaccording to the invention;

FIGS. 4A-C are flowcharts showing in further detail some of the steps inone embodiment of a method according to the invention;

FIG. 5 is a flow chart showing in further detail some of the steps inanother embodiment of a method according to the invention;

FIG. 6 shows an example of fraud confidence as a function of detectedinconsistencies in one embodiment of the invention;

FIG. 7 is a flowchart showing in further detail some of the steps in yetanother embodiment of a method according to the invention;

FIG. 8 shows an example of fraud confidence as a function of detectedinconsistencies in the detected positions of a barcode or security code;

FIG. 9 is a flowchart showing in further detail some of the steps of ananother embodiment of a method according to the invention;

FIG. 10 is a flowchart showing in further detail some of the steps ofyet another embodiment of a method according to the invention;

FIG. 11 is an illustration of an embodiment including a light source, abeam splitter and a retroreflector configured to allow a barcode camerato operate to detect shapes; and

FIG. 12 is a block diagram illustrating the various components of areverse vending machine configured to implement an embodiment of theinvention.

DETAILED DESCRIPTION

In the following description various examples and embodiments of theinvention are set forth in order to provide the skilled person with amore thorough understanding of the invention. The specific detailsdescribed in the context of the various embodiments and with referenceto the attached drawings are not intended to be construed aslimitations. Rather, the scope of the invention is defined in theappended claims.

In the exemplary embodiments, various features and details are shown incombination. The fact that several features are described with respectto a particular example should not be construed as implying that thosefeatures by necessity have to be included together in all embodiments ofthe invention. Conversely, features that are described with reference todifferent embodiments should not be construed as mutually exclusive. Asthose with skill in the art will readily understand, embodiments thatincorporate any subset of features described herein and that are notexpressly interdependent have been contemplated by the inventor and arepart of the intended disclosure. Explicit description of all suchembodiments would, however, not contribute to the understanding of theprinciples of the invention, and consequently some permutations offeatures have been omitted for the sake of simplicity.

Reference is now made to FIG. 1, which illustrates in a perspective viewa reverse vending machine 10 consistent with the principles of theinvention. The machine can be located for example in a store thataccepts receipt of returnable items and positioned such that it iseasily accessible to customers with returnable items, and also such thatreturnable items can be conveniently stored at the rear of the machine,or in a location to which they can be easily transported from the rearof the machine, either automatically or manually.

The front of the machine includes an opening 12 into which returnableitems can be entered by the customer. Also provided is a display forproviding messages to the customer and an input device allowing thecustomer to enter simple commands, for example indicating that thecustomer has entered all their returnable items. As illustrated in FIG.1, the display and the input device may be combined in the form of atouch screen 14. Alternatively, the display and the input device may beseparate devices. The front of the machine 10 may also include aprinting device 16 from which a receipt may be delivered to thecustomer. However, alternative ways of providing the customer with areceipt can also be contemplated, including transmission of anelectronic receipt, over a wireless or wired network, to be received byan electronic device such as a cellphone or smartphone in the possessionof the customer. The electronic receipt may also be sent directly to acheckout counter, or in the form of electronic payment to the customer'saccount. The customer may also be invited to select a charity to whichthe value of the returned items can be donated, using the input devicefunctionality of the touch screen 14.

The machine 10 may also include a loudspeaker 18 or some other form ofaudible or visual alarm that can be used to issue notifications to thecustomer or to an operator for example in the case of a malfunction,storage capacity overflow or some other issue that needs attention.

When a customer enters a returnable item into the reverse vendingmachine 10, the item must be recognized, its authenticity verified andthe appropriate value must be determined. FIG. 2 shows a cross sectionalview of the most important components along the path traveled by areturnable item that has been inserted into the machine 10. When an itemis returned it is entered through the opening 12 into a chamber 20.Inside the chamber there is provided a conveyor 22 capable oftransporting the item from the opening 12 at the front of the machine 10to the rear of the machine 10 where it can be stored or subject tofurther processing such as for example sorting, further transportation,and destruction. In other embodiments the conveyor 22 may be configuredto transport the item in other directions or to deliver the returneditem to additional transportation means or destruction means configuredto transport the returned item to storage further away from the reversevending machine or to be destroyed or compacted. It is also consistentwith the principles of the invention to exclude the conveyor and totransport returned items by some other means, for example manually or bygravity (free fall or down a chute or slide).

Traditionally, a returned container is observed by one or more cameras24 while it is transported through the chamber 20, and the images areanalyzed electronically in order to determine the authenticity of thecontainer. The images from camera or cameras 24 are typically used toanalyze the shape of a container, and for simplicity this camera may bereferred to as shape camera 24 in this specification.

Later developments have introduced barcode readers or other devices forrecognizing markings on the containers. A number of different types oftechnologies for barcode readers are known in the art, but the principalfeatures shared by most of them include a light source and a lightsensor. The light source can for example be light emitting diodes(LEDs), lasers or lamps, and the light sensor can be one or morephotodiodes, charge-coupled devices (CCDs) or CMOS, or any other sensortechnology known in the art for use in cameras. According to theexemplary embodiment illustrated in FIG. 2, cameras 26 and light sources28 are arranged in a pattern that allows a barcode or other marking tobe read while the container is being entered into or transported throughthe chamber 20. Alternative configurations include readers that arepositioned inside the chamber 20 or at the end of the conveyor 22, oreven adjacent to the opening 12. Some of these configurations requirethat the container can be rotated while inside the chamber in order forany barcode or other marking on the container to become entirely visibleto the barcode reader. This has for example been implemented as part ofthe functionality of the conveyor 22. In other embodiments the cameramay be arranged such that the barcode can be read while the container isbeing entered into the chamber 20, but before it is brought to rest onthe conveyor 22.

For ease of reference, camera or cameras 26 may be referred to asbarcode camera 26 hereinbelow. It must be understood that thedesignations of shape camera and barcode camera does not imply that theimages obtained from these cameras are exclusively or always used toanalyze shape or read barcodes, as will become apparent from thefollowing description.

In some embodiments of the invention the reverse vending machine mayalso receive one or more additional sensors 30, which for example may bebased on ultrasound or laser technology.

Even though the returnable fee or deposit for each individual containeris relatively small, large scale fraud is possible, and an importantchallenge in reverse vending is to detect and prevent fraud attempts. Inthis context a fraud attempt is an attempt at manipulating a reversevending machine to accept a container that normally would be rejected,to manipulate the machine to register the same container repeatedly, orfinding some other way to receive a return fee without having provided acorresponding number of returnable containers. In general, fraudattempts constitute something more sophisticated than simply presentinga non-returnable container, perhaps through error. Consequently,anti-fraud measures go beyond simply recognizing and distinguishingreturnable containers from non-returnable containers, and include stepstaken to detect more sophisticated attempts at deceit. However, this isnot meant to imply that the algorithms or methods used to detect fraudcannot also contribute to correct identification and classification ofreturnable and non-returnable containers.

Traditionally, reverse vending machines have been configured to acceptreturned items if they can be properly categorized, based oncharacteristics such as shape, weight and barcode. Proper frauddetection has been absent, or limited to e.g. detection of foreignobjects, such as a hand, present in the chamber 20 where the item isviewed by the shape camera 24.

Fraud detection is, of course, not necessary if a returned item cannotbe recognized and properly classified as a returnable item in the firstplace; in such a case the item will not be accepted anyway.Consequently, fraud detection implies that a returned item is notaccepted (and possibly that an alarm is triggered) despite the fact thatthe item is properly classified as a returnable item. The presentinvention is based on the realization that it may be possible to deceivea reverse vending machine to accept non-returnable containers if theonly fraud detection is the detection of foreign objects by the shapecamera, and that the likelihood of success for a particular deceptionscheme may depend on the configuration of the reverse vending machine.Consequently, the need for sophisticated detection methods may alsodepend on the configuration of the machine. Many deception schemes donot require use of foreign objects, or the objects may be so small thatthey cannot be detected by traditional shape cameras and their attendantimage processing and classification algorithms. However, most fraudattempts will involve spatial or temporal inconsistencies that can bedetected if the necessary images are provided and are properly analyzed.

According to the present invention this can be accomplished by obtainingtwo or more images, at least one of which is an image of a returneditem. The images can then be subjected to feature extraction algorithmsin order to extract at least two features from the images, and adetermination can be made based on any inconsistency in the position ormotion of one of the features with respect to the other feature.

It has furthermore been realized that different features can beindicative of different types of fraud attempts, and consequently, thatdifferent feature extraction algorithms can be used depending on whattype of fraud one is attempting to detect. It is also possible tocombine different types or classes of feature extraction algorithms anddifferent methods for obtaining the two or more images in order to beable to detect a wide range of different fraud attempts. The presentinvention is therefore not limited to a selection of only one of theembodiments described below. To the contrary, any or all of the methods,algorithms, arrangements and configurations described below can inprinciple be combined in any one reverse vending machine, and they canbe configured to look for inconsistencies that are strictly based on thedetermination of position or motion, or any combination of these. It isfurthermore contemplated to use algorithms that each result in aparticular confidence score and to combine the confidence score obtainedby the various detection and determination algorithms to obtain acombined, or final, confidence score which determines if a fraud attempthas been detected or not.

Reference is now made to FIG. 3, which illustrates in a flow chart thevarious steps of a method consistent with the principles of theinvention. In some embodiments of the invention one or more referenceimages of the chamber 20 without any returned item present may then beperformed in step 301. The reference image or images are stored in amemory of the device. The memory may be a hard drive or any other typeof non-volatile memory, or even a volatile memory, such as for examplethe random access memory (RAM) of an image processing unit. The memorymay be part of the reverse vending machine as such, or it may reside ina different computer accessible to the reverse vending machine over acomputer network.

In a following step 302 a returned item is received through an opening12 in the reverse vending machine. In a next step 303, one or moreimages of the returned item are obtained as the returned item is movingthrough the opening 12, is present in the chamber 20, is moving throughthe chamber 20, or is moving out of the chamber 20. The actual positionof the returned item while the image or images are obtained may dependon the choices of configuration and algorithms in a particular design,and the invention as such is not limited to any particular selection inthis regard. The image or images may be obtained using one or morecameras, as described in further detail below.

After at least one image has been obtained in step 303, the processmoves on to step 304 where additional sensor data is obtained usingsensor 30. This step is not present in all embodiments of the invention,as will be explained in further detail below.

In a following step 305, a first feature extraction algorithm is appliedto a subset of the images obtained in step 303 in order to extract afeature from one or more of the images that have been obtained. Itshould be noted that the first feature extraction algorithm may beapplied to only one image, to several images, or to a composite imageresulting from preprocessing of several images prior to the applicationof the first feature extraction algorithm. It should also be noted thatthe process of obtaining images in step 303 and sensor data in step 304may continue while the first extraction algorithm is applied in step305, for example in order to provide additional images to be subjectedto the first extraction algorithm in step 305, to provide a secondsubset or additional sensor data to be subjected to the second featureextraction algorithm in step 306, or both. The term subset is intendedto cover one, some or all of the images that are made available to theimage processing methods, depending on design choices and choice offeature extraction algorithms in any particular embodiment.

In step 306 a second feature extraction algorithm is applied to a secondsubset of images or to data received from an additional sensor 30. Ifthe second feature extraction algorithm is applied to a second subset ofimages, the images belonging to the two sets may have been obtainedsubstantially at the same time, for example using different cameras, orthey may have been obtained at different points in time. In the lattercase, the images subjected to the second feature extraction algorithmmay have been obtained after the images subjected to the first featureextraction algorithm. However, the images subjected to the secondfeature extraction algorithm may also have been obtained prior to theimages subjected to the first algorithm. For example, if a particularfeature has been extracted by the first feature extraction algorithmfrom a current image, the second algorithm may search for the samefeature in a previous image.

It should be noted that the first and the second algorithms are twodifferent algorithms, and in the terminology adopted in thisspecification using the same algorithm twice in order to find twodifferent features, for example in two different images, would not be ause of a first and a second algorithm, but the use of a first algorithmtwice. However, the first and the second algorithm may have somefeatures in common. For example, both algorithms may use edge detection,or both algorithms may use computation of autocorrelation.

In a next step 307, the results of the feature extraction algorithms arecombined or compared, or otherwise processed, in order to determine ifthere are any inconsistencies that may indicate that the reverse vendingmachine is subjected to a fraud attempt. The determination may be basedon the relationship between the results of the two feature extractionalgorithms in order to determine at least a position or a motion of onefeature relative to another, which is inconsistent with a normal returnof a regular returnable item. A position may for example be the relativeposition of two objects (or features indicative of the presence of twoobjects) detected substantially at the same time. A motion may be theposition of one object (feature) relative to the position of anotherobject (feature) at two different points in time, indicating for examplethat two objects that should have been part of the same larger object,such as a returnable item, are not since they are moving independentlyinside or through the chamber 20.

If it is determined that an inconsistency is detected, as indicated bythe branch point 308, the process moves on to step 309 where aconfidence level of fraud detection is calculated. In some embodimentsof the invention only one scheme for fraud detection is used, anddetermination is binary. In other words, the confidence level is either0 when no inconsistency has been detected (corresponding to the NObranch of decision point 307), or the confidence level is always set to1 in step 308, and a positive determination of a fraud attempt is madefor any inconsistency detected. Alternatively, the confidence level isset somewhere between 0 and 1 depending on the degree of inconsistencydetermined in step 307. The resulting level of confidence can then becombined with the result of other detection algorithms in step 309 for aresulting total confidence level. The additional fraud detectionalgorithms or methods may be variants of the present invention, asdescribed in further detail below, they may be different fraud attemptalgorithms that in themselves are not part of this invention, or theymay be a combination of both.

Those with skill in the art will understand that the illustration ofbranch point 308 and the conditional execution of calculation step 309is conceptual, and not intended to be interpreted as a limitation.Indeed, the comparison step 307, the branch point 308 and thecalculation step 309 may for example be one routine that calculates avalue representing the confidence level for fraud detection continuouslyas images and sensor data are received, without any determination beingmade regarding whether or not such a calculation should take place. Inother words, the comparison may be a calculation of a confidence. Inother cases, it may be necessary to actually find the sought forfeatures to actually compare them in step 307, and to actually determinethat some form of inconsistency is observed before invoking an algorithmfor calculating a confidence level. The choices made in this respect areleft as design choices, and may be determined based on for examplewhether the sought after features always are present, or only in specialcases, on the cost in terms of computing power to perform the varioussteps, and other design choices.

It should also be noted that confidence levels of between 0 and 1 is anarbitrary choice, and that alternative ways to score the likelihood, orconfidence, of fraud detection can be contemplated without departingfrom the principles of the invention. In some embodiments of theinvention, several methods of fraud detection all result in a score of anumber of points that are added together in step 310 to a total score.The decision in step 311 to accept or reject the returned item, andpossibly take other action such as issuing an alarm signal, can then bebased on one or more threshold values for this total score.

After the resulting fraud confidence has been determined the returneditem has been accepted or rejected in step 311, the process is finishedwith respect to that particular set of images (and sensor data, ifapplicable), but it may be repeated for the next set. The next set maystill relate to the same returned item until the item leaves the chamber20 (depending on which features are compared, where the cameras aremounted etc), or they may relate to a subsequent item. It is consistentwith the principles of the invention to process data relating to morethan one item at the same time. For example, a first item may be on itsway through the chamber 20 while a second item is being inserted throughthe opening 12. If both items are in view of cameras and/or sensors in amanner that provides the necessary data for performing the steps of theinvention for both items in parallel, this is within the scope of theinvention. Also, if more than one type of inconsistency is sought for,as discussed with respect to step 310, the various feature extractionand detection algorithms may be executed sequentially or in parallel,depending on configuration and design choices in each case.

A word on terminology may be appropriate at this point. While thefollowing definitions are not intended to be interpreted as limitationson the scope of the invention, they will be adopted in the descriptionof the various embodiments in order to facilitate clarity andunderstanding. As such, the term returnable item will be used to referto an item that should be accepted by the reverse vending machine and acorresponding refund or deposit should be credited towards the personreturning the item. A returnable item will typically be an empty bottle,can or some other container, but the invention is not restricted only tothose types of items. A returned item is an item which is being returnedand for which it remains to be determined whether a fraud attempt isbeing made or not. In other words, a returned item may or may not be areturnable item. An object is any particular thing or part of a thingthat is being inserted into the reverse vending machine or that ispresent in the chamber 20. An object may be a returned item, a part of areturned item, or a foreign object. A feature is anything that can beobserved in or extracted from an image or a plurality of images or fromother sensor data using a feature extraction algorithm. A feature may beindicative of the presence of an object or part of an object. Positionwill be used to refer either to the position of an object in the chamber20, or the position of a feature in an image. By way of example, butwithout limitation, a first feature may be representative of thepresence, position or motion of a bar code, a security mark or an objectwith a particular attribute (e.g. particularly thin, having a particularshape etc), and a second feature may be representative of the presence,position or motion of a returned item as a whole, a part of a returneditem (e.g. the bottom or the top). Any combination of these and otherfeatures consistent with the terminology outlined above are intended tobe covered by this terminology.

FIG. 4 is a flow chart illustrating further details of an embodiment ofthe invention. This embodiment is based on optical correlation betweenat least two images in order to detect motion inside the return chamber20 that is inconsistent with the normal operation of a vending machinewhen a returnable item is received. According to this particularembodiment there is no need for any reference images, and the processstarts in a step corresponding to step 302 in FIG. 3 when a returneditem is inserted into the reverse vending machine. In a following step,corresponding to step 303 in FIG. 3 two or more images of the returneditem are obtained. Typically, a continuous sequence of images areobtained such as for example from one or more video cameras and thelikelihood of fraud, expressed as a confidence level, may be higher ifinconsistencies are detected in images obtained from more than onecamera. However, without loss of generality, the following discussionwill assume that the process is performed on two images obtained atdifferent points in time, and those with ordinary skill in the art willrealize that the two images may be the only two images obtained, theymay be part of a sequence of images obtained while the returned item ismoving through a particular section of the path from the opening 12 andthrough the chamber 20, or even the entire path, or images may beobtained continuously even when no returned item is present. It is alsoconsistent with this embodiment of the invention to obtain several“parallel” sequences of images obtained from a plurality of cameras.

In a sequence of steps 405X corresponding to the first featureextraction algorithm step 305 in FIG. 3, a first image is subjected tothe first feature extraction algorithm in order to identify aninteresting region in the first image. An interesting region can bedefined as a region which can be reliably tracked from one frame to thenext, i.e. which can be found in a second image by a second featureextraction algorithm. In order to be suitable the region should be asunique as possible within the image, and it should have a reasonablysharp correlation peak. Considerations that should be taken into accountwhen searching for an interesting region include that the region shouldnot correlate with other regions of the image in order to avoidincorrect tracking. However, the correlation should not be so sharp thatsubtle changes in illumination or the resolution of the camera obtainingthe images can result in difficulties in tracking the feature from oneimage to another. Corners, and squares in particular, are ideal featuresfor this type of region identification.

In step 4051 edge detection along two axes is performed in order todetect corners. It is within the scope of the invention to detect otherfeatures, but an advantage of edge detection along two axes is thatcorners are easy to track from frame to frame, and that the algorithmfor detection of barcodes, which is typically implemented already inmost reverse vending machines, can be used also to perform this edgedetection. The edge detection results in two binary images, one for eachdirection. In a following step 4052 these two edge images aredownsampled and the first image is decimated in order to create a lowresolution image. Decimation is a two step process including low-passanti-aliasing filtering followed by downsampling. In an exampleconsistent with this embodiment of the invention the low pass filter isa simple averaging filter, and the image is downsampled with a factor of8 in each axis, from an original image resolution of 296 by 752 pixelsto a resulting resolution of 37 by 94 pixels. Other resolutions anddecimation factors are, of course, consistent with this embodiment ofthe invention, as will be readily understood by those with ordinaryskill in the art. The edge images and the original first image aredownsampled to the same resolution.

After the edge images have been downsampled, they may be combined instep 4053 using an AND operation. It may now be necessary to removeregions of the mask where the original image frame was close tosaturation or where the camera's field of view is obscured by internalparts of the reverse vending machine, for example rollers or LEDs. Theresulting mask may now be used in order to identify regions that haveboth vertical and horizontal edges. Finding edges along any twosubstantially orthogonal edges is within the scope of the invention,however, and the terms vertical and horizontal are chosen forconvenience.

In a next step 4054 the search for interesting regions is performed.According to an exemplary embodiment an 8-by-8 square in the decimatedimage is used for the correlation. For each point in the downsampled ANDmask with the value 1, a corresponding pixel in the decimated image isselected, and an 8-by-8 square with that pixel in its center is used forthe correlation. This square is correlated with corresponding squaresshifted a number of pixels to each side. Squares with a small peak topeak range of the pixels it includes, for example less than 64, may bediscarded, since they typically will generate poor quality correlationsand they will most likely be images of the background and not of anyobject in the chamber 20. In an exemplary embodiment, correlation isperformed by correlating the square with a corresponding square shiftedone pixel in each cardinal direction (North, South, East, West—or up,down, right, left) within the same frame. The results should be fairlyhigh, for example with an average value for the four correlation valuesthat is higher than 0.5. Otherwise the precision required when searchingfor the same feature in a different image may become too high. A furtherautocorrelation may then be performed, this time while moving theshifted square four pixels. The results of these correlations should below, for example with a maximum value for the four correlation valuesnot higher than 0.5, otherwise there may be too many acceptablecorrelations in the wrong parts of other images.

If the square passes these tests it contains a feature with a reasonablysharp correlation peak, and it may then be subject to a scoringalgorithm. Before scoring, the eight neighboring pixels of the value 1point in the AND mask may be set to zero in order to avoid evaluatingpoints that are too close or overlap. According to some embodiments, ascoring algorithm may be calculated as the peak-to-peak range, plus 100times the sum of the four nearby correlations (shifted e.g. 1 pixel),minus 100 times the sum of the four far-away correlations (shifted e.g.4 pixels). Other scoring algorithms are, of course, within the scope ofthe invention.

This process is repeated with a new 8-by-8 square for each value 1 pointin the AND mask, and in step 4055 the four highest scoring regions areused as the extracted features from the first algorithm. In otherembodiments fewer or more than four interesting regions are selected.

According to the process described above, edge detection is performedbefore downsampling, which is performed before the creation of the ANDmask. In alternative embodiments, the sequence of steps may bedifferent. In particular, edge detection does not have to be performedat full resolution, but can instead be performed on the decimated image,in which case it is not necessary to downsample the resulting edgeimages. They will instead already have the same resolution as thedecimated image.

In a sequence of steps 406X corresponding to the second featureextraction algorithm step 306 in FIG. 3, a second image is subjected tothe second feature extraction algorithm in order to search for regionsin the second image that correspond to the interesting regions in thefirst image. The second image may be a previous image that is alreadystored in memory in the reverse vending machine, or the process maywait, if necessary, until a later image is obtained and search for theinteresting regions in that image. Whether the second image has alreadybeen obtained, is obtained while the first image is processed, or isobtained after the first image has been processed and it has beendetermined that qualifying interesting regions actually are present, theprinciples remain the same, and conceptually the second image isillustrated in FIG. 3 as being obtained in step 303.

In step 4061 the second image is decimated in a process corresponding tothat which is performed on the first image. In a subsequent step 4062the decimated image is subjected to the actual search. This can be donebased on an estimate of the velocity with which the returned item ismoving through the reverse vending machine. Using this estimate it canbe determined where in the second image the four interesting regionsextracted from the first image can be expected to be found. For each ofthe four regions (or however many regions a particular embodiment of theinvention dictates) a corresponding feature is searched for bycorrelating the feature extracted from the first image (an 8-by-8 block)with a corresponding 8-by-8 block in the estimated position. In mostembodiments the search will first be performed in a region surroundingthe estimated position, for example for 16 8-by-8 blocks with theircenter in a 4-by-4 grid of pixels.

If it is determined in step 4063 that a sufficiently good correlation isfound, the velocity estimate can be updated in step 4064 based on howfar away from the estimated position the best correlation was found. Ifmore than one camera is used, several velocity estimates may bemaintained, one for each camera, since different cameras may view thereturned item from different perspectives and distances. Velocityestimates may also be based on other data, for example a differentcamera such as shape camera 24, an additional sensor 30, the velocitywith which the conveyor 22 transports the returned items, etc. Apositive match inside the region first searched is not in itself anindication of fraud, but the velocities estimated for the four regionsare further processed in step 307 and the following steps in FIG. 3.

If it is determined in step 4063 that none of the squares in theselected regions correlate well enough, the search may be expanded instep 4065 in further 4-by-4 regions in the four cardinal directions, byexpanding the original 4-by-4 region with one pixel in each direction,or in some other convenient manner. If this still does not produce apositive match, the search can be expanded even further and acoarse-grid search can be performed using a lower threshold initially,and then home in around successful correlations.

If a good correlation is found in the second image, a velocity for thefeature, and thereby the velocity for a corresponding object present inthe chamber 20, can be estimated in step 4066. If this estimate deviatestoo much from the estimated velocity used to initiate the search in step4061, or from some other reference velocity, this in itself can be anindication of fraud. This may be the case for example if the estimatedvelocity is negative, indicating that there is an object movingbackwards in the chamber 20. Also, if a feature is missing altogether,this may indicate that an object is travelling suspiciously fast or isconcealed in some manner. Such a condition may contribute towards thefinal confidence level calculated in step 309 in FIG. 3.

When a velocity can be estimated, perhaps from multiple regions, theestimated velocity for that camera can be updated in step 4067,corresponding to the update performed in step 4063.

In a sequence of steps 409X corresponding to step 309 of FIG. 3, a totalscore is calculated, representative of how likely it is that a fraudattempt is in progress based on the results of the steps describedabove. This score is referred to as the confidence level.

In a first step 4091 it is determined whether a barcode mid-guard hasbeen detected. If this is not the case the process moves on to step 4092and exits. If other detection algorithms are operated concurrently, theresults of these may still be taken into consideration in a stepcorresponding to step 310 in FIG. 3, but for the purposes of thisparticular embodiment, no contribution to the confidence level is madefrom the process illustrated in FIG. 4 before a barcode mid-guard hasbeen detected. Of course, the invention is not limited in this respect,and it is consistent with the principles of the invention to proceed tostep 4093 even if no midguard has been detected.

In step 4093 a score can be calculated based on the estimated velocitiesfor the features that were detected in the process described above.Several possibilities are within the scope of the invention. The scoremay for example be based on the feature with an estimated velocity thatdeviates the most from the current estimate of the velocity with whichthe returned item is moving. The score may also take into considerationany difference in estimated velocity for the detected features, i.e.instead of or in addition to doing a comparison with the estimatedvelocity for the returned item, the internal consistency of the movementof several features may be considered. The score may then be increasedby any results from step 4065, for example a feature travelling with anegative velocity or not being found in the search at all.

In a next step 4094 the score is again increased if a conditionindicating fraud was detected in a previous execution of the process,indicating that the suspicious condition is present over time. Forexample, the score may be doubled for each frame that indicates a fraudattempt that was also indicated in a previous frame for that camera.

In step 4095 the score may again be increased based on parallel results.For example, in an embodiment with several cameras the score may beincreased in proportion with the number of cameras that detect the sameindication of fraud.

Finally, in step 4096 the score may be increased depending on where inthe chamber 20 the detected feature, or more precisely its correspondingobject, is located. Anomalies may be more likely to occur near theopening 12 without this being a clear indication of fraud, andconversely, anomalies that occur near the end of the chamber 20, whichtranslates to late in the sequence of images obtained of a particularreturned item, are more likely indications of fraud. According to oneembodiment, the score is therefore doubled for images obtained in themiddle third of the sequence of images and tripled for images obtainedin the last third of the sequence.

It will be understood by those with skill in the art that other scoringschemes may be contemplated, and that all, some or none of the methodsfor increasing the score may be used in any combination or sequence.

Reference is now made to FIG. 5, which is a flow chart illustratingfurther details of another embodiment of the invention. This embodimentis based on consistency in the detection of a barcode on a returneditem. In particular, this embodiment attempts to detect barcodes thatmove differently from the returned item along the transport directioninside the reverse vending machine. This may occur for example if avalid barcode is attached to a stick that is inserted into the reversevending machine at the same time as an invalid item is returned. Theintention of the person attempting the fraud is that the barcode camera26 will read the valid code and that the returned item will be acceptedas a result.

According to this embodiment, the process again starts in a stepcorresponding to step 302 in FIG. 3 with the insertion of a returneditem into the reverse vending machine. In a following step 5031 severaloverlapping images of a barcode on the returned item are captured whilethe barcode is visible to the barcode camera(s) 26. Exactly where in thechamber 20 this will be depends on the position of the barcode camera26, which may be left as a design choice. In a next step 5032 aplurality of images of the returned item are obtained from the shapecamera 24.

In a next step 5041 sensor data from sensor 30 are obtained. In someembodiments this step, which corresponds to step 304 of FIG. 3, mayreplace step 5032. Alternatively the data obtained in this step maysupplement the data obtained in step 5032, or step 5041 may be omitted.

In a sequence of steps 505X corresponding to step 305 in FIG. 3,features are extracted from the images obtained from the barcode camerasin step 5031. Barcodes typically include guard patterns in a mannerwhich is well known from the Universal Product Code (UPC) and variantsof UPC such as EAN, JAN and IAN. The guard patterns are a predeterminedpattern of bars and spaces at the left, middle and right of the code.Consistent with the principles of the invention, an attempt is made todetect at least one of the guard patterns of a barcode in a first imagein step 5051. If more than one guard pattern is detected in an image andthey are of the same type and are close to each other, they can bemerged into a cluster of guard patterns. In a following step 5052 asearch for a corresponding cluster is performed in a second image. Thesecond image may be a previous image that is already stored in memory inthe reverse vending machine, or the process may wait, if necessary,until a later image is obtained and the search may then be performed inthat later image. Whether the second image has already been obtained, isobtained while the first image is processed, or is obtained after thefirst image has been processed and the guard pattern has been detectedis conceptually the same in this respect. The step of obtaining thesecond image is part of the image acquisition of step 303 illustrated inFIG. 3.

The degree of match between a guard pattern or cluster of guard patternsdetected in the first and the second image can for example be based onwhether they both include the same type of guard pattern (left, mid,right), that the clusters have similar vertical position in the twoimages and that there is a minimum of match in the digits read from thetwo clusters. Based on these criteria, a match confidence can begenerated. According to one embodiment, a match confidence on a scalefrom zero to one is assigned to all possible cross matching of clustersfrom the first to the second image. Matches with a sufficiently highmatch confidence can then be selected, and in step 5053 the motion ofthe barcode can be estimated based on the displacement of the matchedguard clusters from one image to the next.

The feature extraction algorithms used to detect barcode guards anddigits may already, at least partly, be implemented in the barcodereading algorithm of the return vending machine. However, since it maybe desirable to detect potential code elements (clusters) in all imagesat all times, e.g. to detect labels with codes at all possible positionsin the chamber 20, additional processing may be required compared to therequirements for barcode reading only, since the barcode reading can beterminated as soon as a valid code has been read.

Moving on to step 5061, corresponding to step 306 of FIG. 3, the imagesfrom the shape camera 24 and/or data from the sensor 30 are processedand an estimate of the motion of the returned item is extracted. Thisprocess is not illustrated in detail, but may be based, for example, ondetection of characteristic features of the shape of the returned itemas detected in images obtained by the shape camera, or on velocity orrange determination based on signals issued and received by the sensor30, for example based on well known ultrasound or laser technologies.

In step 5071, corresponding to step 307 of FIG. 3, the determined motionof the barcode and the returned item are compared. The process thencontinues as illustrated in step 308 of FIG. 3. If fraud is indicated,the calculated fraud confidence of step 309 may be based on thedeviation in motion. The confidence may be binary, in the sense that afraud attempt is indicated as ongoing if the deviation is above acertain threshold, and determined to be absent if the deviation is belowthe threshold. In other embodiments the fraud confidence is a functionof the deviation. An example of such a function is illustrated in FIG.6, which shows the fraud confidence as a function of deviation betweenbarcode and returned item motion. The deviation is given as pixel unitsin the barcode images and the confidence score is on a scale going from0 to 10.

The embodiment illustrated in FIG. 7 is an alternative way of detectingbarcode inconsistency. In this embodiment it is not the relative motion,or velocity, of the barcode with respect to the returned item that ismeasured, but the position. The fraud attempt may be the same, forexample that a valid barcode is inserted using a stick or string, andwithdrawn after it has been read by the barcode camera or cameras 26.

Determination of the velocity with which a barcode is moving may in someinstances be difficult or unreliable. As described above, it isnecessary to detect the same guard patterns from one image to the next.For example, a barcode moving independently of the returned item maymove too fast for any reliable estimate of its motion to be determined,for example when it is pulled out again from the reverse vending machineby the person attempting the fraud. In an alternative embodiment, whichmay be used instead of or as a supplement to the embodiment discussedwith reference to FIG. 5, determination of the barcode velocity is notnecessary. Instead, it is necessary to know approximately where on areturned item a valid barcode can be expected, taking into account thatdifferent items have barcodes located in different positions, and thatsome items may include several valid barcodes located in differentpositions on the item.

It is possible to maintain a database in the reverse vending machine, orone that is accessible to the reverse vending machine, in which thepossible locations of the code on all recognized returnable items arestored. Such a database may be continuously updated online.Alternatively, short term statistics may be generated based on datacollected during a customer session (i.e. the period from a firstreturned item is inserted to the customer indicates that all items havebeen returned by requesting a receipt), or over a predetermined periodof time. These statistics can be based on the assumption that items ofthe same kind, i.e. items bearing the same code, should have the code inapproximately the same position. During a session, the detected positionof a particular code on each item bearing that code can be collected,and a fraud attempt can be reported if the same code is detected atseveral different positions on returned items.

Again the process starts as in FIG. 3 when a returned item is insertedinto the reverse vending machine in step 302. In step 7031, at least oneimage of the barcode is obtained, and in step 7032 at least one image ofthe returned item is obtained. In addition, or as an alternative to step7032, sensor data is obtained in step 7041. In step 7051 guard patternsin the barcode image are detected, and the detected patterns are used instep 7052 to determine the position of the barcode in the chamber 20. Ina following step 7061 the position of the returned item is determinedbased on the at least one image of the returned item obtained from theshape camera 24 in step 7032 and/or from the sensor data obtained instep 7041. It will be understood that the position of the barcode andthe returned item must be determined at substantially the same time.

In a sequence of steps 707X corresponding to step 307 of FIG. 3, theposition of the barcode is compared to the position of the returned itemand the consistency of the barcode position on the returned item isdetermined. In step 7071 the results of the two previous steps arecombined in order to determine the position of the barcode on thereturned item. In step 7072 the position that has been established asstatistically valid, based on previously returned items, is retrieved inorder to be compared with the position determined in step 7071. Inalternative embodiments this position may be retrieved from a databaseof valid barcode positions, as described above.

In step 7073 the consistency of the barcode position relative to theposition of the returned item, as determined based on the collectedstatistics, is determined. If there is no inconsistency, the processmoves from branch point 7081 to step 7082 where the collected statisticsare updated with the position detected in this instance. If, however,the position is inconsistent with previous detections, the processproceeds to step 7091, corresponding to step 309 in FIG. 3, where aconfidence of fraud detection is calculated. The confidence can be afunction of the deviation between the statistically determined validposition and the currently determined position. FIG. 8 illustrates anexample of fraud confidence as a function of deviation in positionbetween the current and earlier detections of a code. The deviation ismeasured as pixels in the barcode images, and the fraud confidence is ona scale from 0 to 10.

It should be noted that several positions may be valid for differentitems bearing the same code, such that it may be pertinent to collectstatistics on position even if an inconsistency is detected (e.g. priorto branch point 7081, or as part of step 7091 as well as step 7082). Forexample, all positions that have already been detected once may beconsidered as acceptable if they are repeated, or the average positionof any cluster of positions within a certain range from each other maybe considered acceptable as soon as the cluster includes a certainnumber of detected positions. It will be understood that the firstinstances inside such a cluster, i.e. the positions detected beforeenough instances have been detected to determine that the position isassumed to be a valid one, will be detected as inconsistent, but theymay not contribute enough to the accumulated fraud confidence in step310 to result in a rejection of the returned item or an issuance of anyalarm.

A further embodiment of the invention is based on the realization thatmany fraud attempts include foreign objects that are difficult to detectbecause they are very thin or because they appear as part of thereturned item due to close proximity. Consequently, detection of thinobjects or edges and their proximity or movement relative to thereturned item can be used as an indication of a fraud attempt.

Schlieren photography is a visual process that is well known forvisualization of the flow of fluids of varying density. However, theprocess is also useful for detection of shapes, edges and thin objects,and it has previously been suggested to use Schlieren cameras or similarsystems as the shape camera 24 in reverse vending machines. U.S. Pat.No. 5,898,169, which is hereby incorporated by reference in itsentirety, describes a device where containers such as bottles passbetween a retroreflector and a Fresnel lens. Light from a light sourceis passed through a beam splitter and the Fresnel lens, reflected by theretroreflector back through the Fresnel lens, and directed from the beamsplitter to a camera. The camera will then see objects such as bottlespassing through the area between the retroreflector and the lens as darksilhouettes against a bright background. Another Schlieren-like systemfor determining the characteristics of e.g. a bottle that is placed inan observation area is described in U.S. Pat. No. 7,633,614, which ishereby incorporated by reference in its entirety. The description of thepresent embodiment will assume that least one barcode camera 26 iscapable of operating in a Schlieren-like mode or in some other modefacilitating observation of shapes, edges and thin objects, as will bedescribed in further detail below. In addition the shape camera 24 maybe part of a Schlieren-like system.

A process implementing this embodiment is illustrated in a flow chart inFIG. 9. The initial step of obtaining a reference image while thechamber 20 is empty is performed using a barcode camera in Schlierenmode. This step can be performed while the reverse vending machine is inan idle mode, and the stored images may be regularly updated byrepeating this step at regular intervals. In a next step a returned itemis received by the reverse vending machine. These steps are illustratedas step 301 and step 302 in FIG. 3. The process then proceeds to step9031 where at least one image is obtained using the same camera as forthe reference image, for example the shape camera 24 or a barcode camera26 operating in a Schlieren mode. In a next step 9032 at least one imageof the returned item is obtained using the shape camera 26. In additionto, or instead of, the image obtained by the shape camera, sensor datamay be obtained by the sensor 30 in step 9041.

In a next step 9051 the image obtained using the barcode camera 26 isprocessed. The obtained image can be processed by dividing it by thereference image on a pixel by pixel basis in order to calculate atransmission coefficient at each pixel. Alternative processing methodsinclude subtracting the reference image from the obtained image, or anyother suitable foreground-background segmentation method. The resultscan be used to create a processed image where each pixel is eitherclassified as object (1, or “white”) or background (0, or “black”). Inthis processed image, sequences of pixels with the value 1 areclassified as “thin” or “normal” based on the length of the sequenceacross the direction of movement for returned items, and adjacent “thin”pixels are grouped together to represent “thin objects”. In order to beclassified as “thin” the sequence must be shorter than a thresholdvalue. This threshold value may be a tuning parameter dependent on theconfiguration of the reverse vending machine and the desired sensitivityof the detection. In some cases it may be required to classify onlyobjects that are less than one or two millimeters thick as “thin”, whilein other cases the designer or operator may desire to classify objectsthat are as thick as perhaps 10 mm or 15 mm. The threshold number ofpixels must be chosen correspondingly, depending on the resolution ofthe images, the positions of the camera with respect to the returneditems, etc. In step 9053, the position of any detected thin object isdetermined. The processing of steps 9051-9053 correspond to theprocessing of step 305 in FIG. 3.

In a next step 9061 the position of the returned item is determinedbased at least either an image obtained using the shape camera 24 instep 9032 or sensor data obtained in step 9041. The position of anydetected thin object relative to the position of the returned item canthen be established in step 9071. Additional processing may also beincluded in order to detect acceptable thin objects such as cap tabs.For example, the fraud confidence for a detected thin object that isclassified as very likely to be a cap tab can be reduced in proportionto its “cap tab score”. This cap tab score can be based on measuredproperties as compared with expected properties for cap tabs (or otheracceptable thin objects) stored in a database or in a file.

In step 9072 the consistency of any detected thin object with the validreturn of an item is determined. The consistency can be based on theposition of the thin object relative to the returned item. If aninconsistency can be determined, the process follows the steps outlinedin FIG. 3 from branch point 308. Different metrics for the calculationof the fraud confidence score in step 310 can be contemplated withoutdeparting from the scope of the invention. For example, the closer thethin object is to the top end of the returned item (e.g. the can orbottle opening) and the more similar its measurements is to the expectedmeasurements of a cap tab, the lower the fraud confidence is determinedto be. Other metrics may include whether the thin object changes itsposition relative to the returned item over time (e.g. indicating thatit moves independently of the returned item), whether it appears to beattached to the returned item or not, and whether it moves toward oraway from the returned item.

The embodiment illustrated in FIG. 10 is similar to the one describedwith reference to FIG. 9, but in this embodiment the first featureextraction algorithm searches for edges along the outline of thereturned item instead of thin objects. In particular, characteristicfeatures of the edge, such as “steps” or discontinuities are tracked. Ifsuch features move differently from the overall motion of the returneditem or simply disappear from one image to the next, this may be anindication of fraudulent activity.

The initial steps are similar to those of the preceding embodiment, inthat a reference image is obtained in step 301 and a returned item isinserted into the reverse vending machine in step 302 (as illustrated inFIG. 3). In step 10031 a plurality of Schlieren images are obtainedusing a barcode camera 26 and in step 10032 a plurality of images of thereturned item are obtained using the shape camera 24. In someembodiments sensor data from sensor 30 obtained in step 10041 replacesor supplements the images obtained using the shape camera 24.

In step 10051 the images obtained in step 10031 are processed usingcorresponding reference images. The resulting composite image is animage where all pixels are classified as either background or object, asdiscussed above. The composite image is further processed in step 10053,where distinguishing features of the edge profile are detected and theirpositions are determined. This can for example be done by calculatingthe 1st derivative of the edge profile, for example by drawing a curvethat traces the edge between object pixels and background pixels andcalculating the 1st derivative of this curve. In step 10061 the positionof the returned item is determined based on images obtained in step10032 and/or sensor data obtained in step 10041.

In step 10071 the motion of any distinguishing features detected in step10053 (i.e. the change in position over two or more images) are comparedwith corresponding changes in position for the returned item as a whole.For example, any peak in the 1st derivative should remain in the sameposition relative to the returned item, i.e. distinguishing featuresshould not display any independent motion relative to the returned itemor disappear. If such a distinguishing feature does move independentlyfrom the returned item or disappears, this can for example be the resultof a label with a barcode being attached to a stick or a string andmoved independently into the chamber 20, or it may indicate that aperson is holding the returned item and reaching into the chamber 20beyond the opening zone 12 where a user is supposed to let go and removetheir hand.

If it can be determined in step 10072 that an indication of fraud may bepresent, i.e. that movement or loss of a distinguishing feature of theitem profile is inconsistent with the normal return of a validreturnable item, the process proceeds with step 308 as described withreference to FIG. 3. The metrics used when calculating the fraudconfidence level, or confidence score, may include dimensions of adistinguishing feature such as the step height where the 1st derivativehas a peak, the position on the returned item, the position in thechamber such that an inconsistency that occurs near the exit of thechamber 20 is more suspicious than an inconsistency that occurs near theopening 12. Other metrics may also be contemplated without departingfrom the scope of the invention. Also, additional calculations may beperformed in order to disregard inconsistencies that can be classifiedas acceptable, for example profile features that can be identified as apartly loose label.

FIG. 11 is an illustration of an exemplary embodiment of how a barcodecamera can operate in a mode that is primarily directed at detectingshapes, edges or thin objects rather than reading barcodes. According tothis embodiment the barcode camera 26 is directed toward the chamber 20as already described with reference to FIG. 2, but in front of thecamera there is a beam splitter 32. A light source 28 is positioned atthe same distance from the beam splitter 32 as the camera 26, such thatoptically speaking, as seen from the chamber 20 the light source 28 andthe camera 26 are in the same position. At the far end of the chamber 20there is a retroreflector which reflects light directly back at thecamera 26. In the embodiment illustrated the retroreflector is a concavespherical mirror. However, other configurations are possible.Alternatively, a flat mirror could be used in combination with a Fresnellens, as described in U.S. Pat. No. 5,898,169.

The result of this configuration is that the image captured by thecamera 26 will show a very bright background, and any object in theimage will consequently be relatively much darker. By processing theimage using an empty reference image showing only the background, thiseffect can be further enhanced, giving the images discussed above.

Reference is now made to FIG. 12, which is a block diagram illustratingthe various components of an exemplary embodiment of a reverse vendingmachine operating in accordance with the principles of the invention. Anumber of light sources 28 are connected to processing unit 40 whichincludes a synchronization module 42. The synchronization module 42 maybe configured to control any required synchronized operation of thelight sources 28, the cameras 24,26, the sensor 30 and the conveyor 22,such as controlling which light source 28 illuminate the chamber when aparticular camera 24,26 is activated, or ensuring that two or morecameras are activated simultaneously or in a particular sequence, asdictated by any of the embodiments described above or combinationsthereof. Data returned from the cameras (or other suitable lightdetectors or light sensors) 24,26, sensor 30 and conveyor 22 arereceived by the processing means 40. Further, the reverse vendingmachine may include a comparison module 44 configured to compare animage detected by the shape camera 24 to recognize a contour image of aparticular returned item and communicating such recognition back to theprocessing unit 40. Input from and output to a user may be received byand controlled by the processor unit 40, respectively, through a displayand input device which may be combined in a touch screen 14, a printer16 and a loudspeaker 18.

The reverse vending machine may also include a bar code reading unit 46and a feature extraction unit 48. Those with skill in the art willrealize that the shape comparison unit 44, the bar code reading unit 46and the feature extraction unit 48 may be implemented as software storedin a storage device (not shown) and configured to be executed by theprocessing unit 40. These units may also be fully or partly implementedas separate processors, graphic processors, field programmable gatearrays (FPGA), application specific integrated circuits (ASIC), or acombination thereof. Furthermore, several of the various algorithms,instructions and hardware implementing these units may operate as partof more than one such unit. For example, algorithms implemented assoftware modules in the bar code reading unit 46, may double as part ofone or more feature extraction functions that are part of the featureextraction unit 48.

Those with skill in the art will also understand that additionalcomponents that are not illustrated in FIG. 12 may be part of a reversevending machine or a system of reverse vending machines. Such componentsmay, for example, include a power supply, a communication interface forcommunication with remote computers or storage units, various storageunits including volatile and non-volatile memory units, databases etc.

1. A method for detecting fraud attempts in a reverse vending machine,the reverse vending machine comprising at least one image sensor and atleast one data processing unit, the method comprising: using said atleast one image sensor to obtain a set of one or more images of areturned item inserted into the reverse vending machine; using at leastone data processing unit to apply a first feature extraction algorithmto extract a first feature from at least a first subset of said set ofone or more images; using at least one data processing unit to apply asecond feature extraction algorithm to extract a second feature from i)at least a second subset of said set of one or more images or ii) datareceived from an additional sensor; and using at least one dataprocessing unit to determine whether a position or a motion of saidfirst feature is inconsistent with a position or motion of said secondfeature.
 2. A method according to claim 1, wherein said second featureextraction algorithm is applied to data received from an ultrasonicsensing device included in said reverse vending machine.
 3. A methodaccording to claim 1, wherein said second feature extraction algorithmis applied to data received from a laser ranging device included in saidreverse vending machine.
 4. The method of claim 1, wherein: obtainingsaid set of one or more images includes obtaining at least two imagesobtained at least two different points in time.
 5. The method of claim4, wherein: said first feature extraction algorithm is used to extract afeature from an image obtained at a first point in time, said featurebeing indicative of a relative position of an object at said first pointin time; said second feature extraction algorithm is used to extract afeature from an image obtained at a second point in time, said featurebeing indicative of a relative position of said object at said secondpoint in time; and said determining includes estimating a velocity ofsaid object based on said relative positions and the time intervalbetween said first and second point in time, and comparing saidestimated velocity with an expected velocity.
 6. The method of claim 5,wherein said expected velocity is any velocity which is positiverelative to the direction a returnable item normally travels through thereverse vending machine, and said determining indicates inconsistentmotion if said estimated velocity is a velocity which is negativerelative to the direction a returnable item normally travels through thereverse vending machine.
 7. The method of claim 5, wherein said expectedvelocity is based on the velocity at which a conveyor arranged totransport returned items through said reverse vending machine isoperating, and said determining indicates inconsistent motion if saidestimated velocity is substantially different from said expectedvelocity.
 8. The method of claim 5, wherein said expected velocity isbased on a determined estimated velocity resulting from the applicationof said first feature extraction, said second feature extraction andsaid determining on a previous set of images, and the method furthercomprises using the result of said determining to update the expectedvelocity prior to the application of said first feature extraction, saidsecond feature extraction and said determining on a subsequent set ofimages.
 9. The method of claim 5, wherein: said first feature extractionalgorithm includes applying edge detection along two axes and, based onsaid edge detection, identifying a region of interest in said firstimage; and said second feature extraction algorithm includes searchingfor a region in said second image which is highly correlated with saidregion of interest in said first image.
 10. The method of claim 9,wherein said first image is obtained at a current point in time and saidsecond image is obtained at a previous point in time.
 11. The method ofclaim 1, wherein: said first feature extraction algorithm is used toextract a first feature indicative of a relative position of a firstobject associated with said first feature from at least two imagesobtained at least two different points in time; said second featureextraction algorithm is used to extract a second feature indicative of arelative position of a second object associated with said second featurefrom at least two images obtained at least two different points in time;and said determining includes estimating relative velocities for saidfirst and said second object based on said relative positions and timeintervals between the different points in time, and indicating aninconsistent motion of said first object relative to said second objectif there is a substantial difference between said first and said secondestimated velocity.
 12. The method of claim 11, wherein said firstfeature extraction algorithm is applied to images obtained from a firstimage sensor and said second feature extraction algorithm is applied toimages obtained from a second image sensor, and where said first featureis representative of a barcode and said second feature is representativeof a shape of a returned item.
 13. The method of claim 1, wherein: saidfirst feature extraction algorithm includes detecting the presence andposition of a thin object in at least one image from said first subsetof images; said second feature extraction algorithm includes detecting aposition of a returned item from at least one image from said secondsubset of images or from said sensor data; and said determining includescomparing the position of the detected thin object relative to theposition of the returned item.
 14. The method of claim 13, whereindetecting the presence and position of a thin object includes using aforeground-background segmentation method to generate a processed image,classifying each pixel in said processed image as either an object pixelor a background pixel, and classifying each object pixel as either“normal” or “thin”.
 15. The method of claim 14, wherein saidforeground-background segmentation method includes dividing a currentimage by a reference image.
 16. The method of claim 1, wherein: saidfirst feature extraction algorithm includes detecting the presence andposition of at least one characteristic feature of the shape of areturned item from at least two images in said first set of images, saidat least two images being obtained at different points in time; saidsecond feature extraction algorithm includes detecting a position of areturned item from at least two images from said second subset of imagesor from said sensor data; and said determining indicates aninconsistency if said characteristic feature has a relative positionwith respect to the overall shape of the returned item which differssubstantially in said at least two images from said first set of images.17. The method of claim 16, wherein detecting the presence and positionof at least one characteristic feature of the shape of a returned itemincludes using a foreground-background segmentation method to generate aprocessed image, classifying each pixel in said processed image aseither an object pixel or a background pixel, and detecting a feature asa peak in the 1st derivative of a curve tracing the edge between objectpixels and background pixels.
 18. The method of claim 17, wherein saidforeground-background segmentation method includes dividing a currentimage by a reference image.
 19. The method of claim 13, wherein saidfirst subset of images are obtained using an image sensor which isconfigured to view a returned item silhouetted against a backgroundincluding a retroreflector, and where a light source and a beam splitterare configured such that said light source and said image sensor areoptically co-located as seen from the retroreflector.
 20. A reversevending machine with an opening through which items can be inserted intothe interior of said machine, at least one image sensor configured toobtain images of at least a part of an inserted item, and at least onedata processing unit, the reverse vending machine comprising: at leastone data processing unit configured to receive a set of one or moreimages of a returned item inserted into the reverse vending machine fromsaid at least one image sensor; at least one feature extraction unitconfigured to apply a first feature extraction algorithm to extract afirst feature from at least a first subset of said set of one or moreimages and apply a second feature extraction algorithm to extract asecond feature from i) at least a second subset of said set of one ormore images, or ii) data received by said at least one data processingunit from an additional sensor; and at least one data processing unitconfigured to determine whether a position or a motion of said firstfeature is inconsistent with a position or a motion of said secondfeature.
 21. The reverse vending machine of claim 20, further comprisingan ultrasonic sensor configured to obtain sensor data from inside saidinterior and provide said data to said at least one data processingunit; and wherein said feature extraction unit is configured to applysaid second feature extraction algorithm to data received from saidultrasonic sensor.
 22. The reverse vending machine of claim 20, furthercomprising a laser ranging device configured to obtain sensor data frominside said interior and provide said data to said at least one dataprocessing unit; and wherein said feature extraction unit is configuredto apply said second feature extraction algorithm to data received fromsaid laser ranging device.
 23. The reverse vending machine of claim 20,wherein said at least one data processing unit is configured to controlsaid at least one image sensor to obtain images at least two differentpoints in time.
 24. The reverse vending machine of claim 23, whereinsaid at least one feature extraction unit is configured to apply saidfirst feature extraction algorithm to an image obtained at a first pointin time and to apply said second feature extraction algorithm to animage obtained at a second point in time; and said at least one dataprocessing unit is configured to estimate a velocity of an objectpresent in said interior based on the relative positions of said firstand said second features, and to compare said estimated velocity with anexpected velocity.
 25. The reverse vending machine of claim 23, whereinsaid at least one feature extraction unit is configured to apply saidfirst feature extraction algorithm and said second feature extractionalgorithm to images obtained at least two different points in time; andsaid at least one data processing unit is configured to estimaterelative velocities for a first and a second object associated with saidfirst and said second feature, respectively, and to indicate aninconsistent motion of said first object relative to said second objectif there is a substantial difference between said first and said secondestimated velocity.
 26. The reverse vending machine of claim 25, whereinsaid at least one image sensor includes at least one first image sensorwith a relatively narrow field of view in order to capture images ofbarcodes attached to returned items at least partly present in saidinterior, and at least one second image sensor with a relatively widefield of view in order to capture images of the shape of returned itemsat least partly present in said interior.
 27. The reverse vendingmachine of claim 20, further comprising: a retroreflector, a beamsplitter and a light source; and wherein at least one image sensor isconfigured to view a returned object silhouetted against saidretroreflector, and said light source and said beam splitter areconfigured such that said light source and said image sensor areoptically co-located as seen from the retroreflector.
 28. The reversevending machine of claim 20, wherein the feature extraction unitcomprises one or more components chosen from the group consisting of: aset of computer program instructions stored on a computer readable mediaand configured to enable the at least one data processing unit toperform feature extraction functions, a field programmable gate array(FPGA), and an application specific integrated circuit (ASIC).